<!DOCTYPE html>

<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <meta http-equiv="x-ua-compatible" content="ie=edge">
    
    <title>3.4.1. 基础结构演进 &#8212; FunRec 推荐系统 0.0.1 documentation</title>

    <link rel="stylesheet" href="../../_static/material-design-lite-1.3.0/material.blue-deep_orange.min.css" type="text/css" />
    <link rel="stylesheet" href="../../_static/sphinx_materialdesign_theme.css" type="text/css" />
    <link rel="stylesheet" href="../../_static/fontawesome/all.css" type="text/css" />
    <link rel="stylesheet" href="../../_static/fonts.css" type="text/css" />
    <link rel="stylesheet" type="text/css" href="../../_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="../../_static/basic.css" />
    <link rel="stylesheet" type="text/css" href="../../_static/d2l.css" />
    <script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
    <script src="../../_static/jquery.js"></script>
    <script src="../../_static/underscore.js"></script>
    <script src="../../_static/_sphinx_javascript_frameworks_compat.js"></script>
    <script src="../../_static/doctools.js"></script>
    <script src="../../_static/sphinx_highlight.js"></script>
    <script src="../../_static/d2l.js"></script>
    <script async="async" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
    <link rel="index" title="Index" href="../../genindex.html" />
    <link rel="search" title="Search" href="../../search.html" />
    <link rel="next" title="3.4.2. 任务依赖建模" href="2.dependency_modeling.html" />
    <link rel="prev" title="3.4. 多目标建模" href="index.html" /> 
  </head>
<body>
    <div class="mdl-layout mdl-js-layout mdl-layout--fixed-header mdl-layout--fixed-drawer"><header class="mdl-layout__header mdl-layout__header--waterfall ">
    <div class="mdl-layout__header-row">
        
        <nav class="mdl-navigation breadcrumb">
            <a class="mdl-navigation__link" href="../index.html"><span class="section-number">3. </span>精排模型</a><i class="material-icons">navigate_next</i>
            <a class="mdl-navigation__link" href="index.html"><span class="section-number">3.4. </span>多目标建模</a><i class="material-icons">navigate_next</i>
            <a class="mdl-navigation__link is-active"><span class="section-number">3.4.1. </span>基础结构演进</a>
        </nav>
        <div class="mdl-layout-spacer"></div>
        <nav class="mdl-navigation">
        
<form class="form-inline pull-sm-right" action="../../search.html" method="get">
      <div class="mdl-textfield mdl-js-textfield mdl-textfield--expandable mdl-textfield--floating-label mdl-textfield--align-right">
        <label id="quick-search-icon" class="mdl-button mdl-js-button mdl-button--icon"  for="waterfall-exp">
          <i class="material-icons">search</i>
        </label>
        <div class="mdl-textfield__expandable-holder">
          <input class="mdl-textfield__input" type="text" name="q"  id="waterfall-exp" placeholder="Search" />
          <input type="hidden" name="check_keywords" value="yes" />
          <input type="hidden" name="area" value="default" />
        </div>
      </div>
      <div class="mdl-tooltip" data-mdl-for="quick-search-icon">
      Quick search
      </div>
</form>
        
<a id="button-show-source"
    class="mdl-button mdl-js-button mdl-button--icon"
    href="../../_sources/chapter_2_ranking/4.multi_objective/1.arch.rst.txt" rel="nofollow">
  <i class="material-icons">code</i>
</a>
<div class="mdl-tooltip" data-mdl-for="button-show-source">
Show Source
</div>
        </nav>
    </div>
    <div class="mdl-layout__header-row header-links">
      <div class="mdl-layout-spacer"></div>
      <nav class="mdl-navigation">
          
              <a  class="mdl-navigation__link" href="https://funrec-notebooks.s3.eu-west-3.amazonaws.com/fun-rec.zip">
                  <i class="fas fa-download"></i>
                  Jupyter 记事本
              </a>
          
              <a  class="mdl-navigation__link" href="https://github.com/datawhalechina/fun-rec">
                  <i class="fab fa-github"></i>
                  GitHub
              </a>
      </nav>
    </div>
</header><header class="mdl-layout__drawer">
    
          <!-- Title -->
      <span class="mdl-layout-title">
          <a class="title" href="../../index.html">
              <span class="title-text">
                  FunRec 推荐系统
              </span>
          </a>
      </span>
    
    
      <div class="globaltoc">
        <span class="mdl-layout-title toc">Table Of Contents</span>
        
        
            
            <nav class="mdl-navigation">
                <ul>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_preface/index.html">前言</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_installation/index.html">安装</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_notation/index.html">符号</a></li>
</ul>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../chapter_0_introduction/index.html">1. 推荐系统概述</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_0_introduction/1.intro.html">1.1. 推荐系统是什么？</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_0_introduction/2.outline.html">1.2. 本书概览</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_1_retrieval/index.html">2. 召回模型</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_1_retrieval/1.cf/index.html">2.1. 协同过滤</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/1.cf/1.itemcf.html">2.1.1. 基于物品的协同过滤</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/1.cf/2.usercf.html">2.1.2. 基于用户的协同过滤</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/1.cf/3.mf.html">2.1.3. 矩阵分解</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/1.cf/4.summary.html">2.1.4. 总结</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_1_retrieval/2.embedding/index.html">2.2. 向量召回</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/2.embedding/1.i2i.html">2.2.1. I2I召回</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/2.embedding/2.u2i.html">2.2.2. U2I召回</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/2.embedding/3.summary.html">2.2.3. 总结</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_1_retrieval/3.sequence/index.html">2.3. 序列召回</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/3.sequence/1.user_interests.html">2.3.1. 深化用户兴趣表示</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/3.sequence/2.generateive_recall.html">2.3.2. 生成式召回方法</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/3.sequence/3.summary.html">2.3.3. 总结</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1 current"><a class="reference internal" href="../index.html">3. 精排模型</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../1.wide_and_deep.html">3.1. 记忆与泛化</a></li>
<li class="toctree-l2"><a class="reference internal" href="../2.feature_crossing/index.html">3.2. 特征交叉</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../2.feature_crossing/1.second_order.html">3.2.1. 二阶特征交叉</a></li>
<li class="toctree-l3"><a class="reference internal" href="../2.feature_crossing/2.higher_order.html">3.2.2. 高阶特征交叉</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../3.sequence.html">3.3. 序列建模</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="index.html">3.4. 多目标建模</a><ul class="current">
<li class="toctree-l3 current"><a class="current reference internal" href="#">3.4.1. 基础结构演进</a></li>
<li class="toctree-l3"><a class="reference internal" href="2.dependency_modeling.html">3.4.2. 任务依赖建模</a></li>
<li class="toctree-l3"><a class="reference internal" href="3.multi_loss_optim.html">3.4.3. 多目标损失融合</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../5.multi_scenario/index.html">3.5. 多场景建模</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../5.multi_scenario/1.multi_tower.html">3.5.1. 多塔结构</a></li>
<li class="toctree-l3"><a class="reference internal" href="../5.multi_scenario/2.dynamic_weight.html">3.5.2. 动态权重建模</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_3_rerank/index.html">4. 重排模型</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_3_rerank/1.greedy.html">4.1. 基于贪心的重排</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_3_rerank/2.personalized.html">4.2. 基于个性化的重排</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_3_rerank/3.summary.html">4.3. 本章小结</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_4_trends/index.html">5. 难点及热点研究</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_4_trends/1.debias.html">5.1. 模型去偏</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_4_trends/2.cold_start.html">5.2. 冷启动问题</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_4_trends/3.generative.html">5.3. 生成式推荐</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_4_trends/4.summary.html">5.4. 本章小结</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_5_projects/index.html">6. 项目实践</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/1.understanding.html">6.1. 赛题理解</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/2.baseline.html">6.2. Baseline</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/3.analysis.html">6.3. 数据分析</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/4.recall.html">6.4. 多路召回</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/5.feature_engineering.html">6.5. 特征工程</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/6.ranking.html">6.6. 排序模型</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_appendix/index.html">7. Appendix</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_appendix/word2vec.html">7.1. Word2vec</a></li>
</ul>
</li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_references/references.html">参考文献</a></li>
</ul>

            </nav>
        
        </div>
    
</header>
        <main class="mdl-layout__content" tabIndex="0">

	<script type="text/javascript" src="../../_static/sphinx_materialdesign_theme.js "></script>
    <header class="mdl-layout__drawer">
    
          <!-- Title -->
      <span class="mdl-layout-title">
          <a class="title" href="../../index.html">
              <span class="title-text">
                  FunRec 推荐系统
              </span>
          </a>
      </span>
    
    
      <div class="globaltoc">
        <span class="mdl-layout-title toc">Table Of Contents</span>
        
        
            
            <nav class="mdl-navigation">
                <ul>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_preface/index.html">前言</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_installation/index.html">安装</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_notation/index.html">符号</a></li>
</ul>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../chapter_0_introduction/index.html">1. 推荐系统概述</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_0_introduction/1.intro.html">1.1. 推荐系统是什么？</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_0_introduction/2.outline.html">1.2. 本书概览</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_1_retrieval/index.html">2. 召回模型</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_1_retrieval/1.cf/index.html">2.1. 协同过滤</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/1.cf/1.itemcf.html">2.1.1. 基于物品的协同过滤</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/1.cf/2.usercf.html">2.1.2. 基于用户的协同过滤</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/1.cf/3.mf.html">2.1.3. 矩阵分解</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/1.cf/4.summary.html">2.1.4. 总结</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_1_retrieval/2.embedding/index.html">2.2. 向量召回</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/2.embedding/1.i2i.html">2.2.1. I2I召回</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/2.embedding/2.u2i.html">2.2.2. U2I召回</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/2.embedding/3.summary.html">2.2.3. 总结</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_1_retrieval/3.sequence/index.html">2.3. 序列召回</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/3.sequence/1.user_interests.html">2.3.1. 深化用户兴趣表示</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/3.sequence/2.generateive_recall.html">2.3.2. 生成式召回方法</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../chapter_1_retrieval/3.sequence/3.summary.html">2.3.3. 总结</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1 current"><a class="reference internal" href="../index.html">3. 精排模型</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../1.wide_and_deep.html">3.1. 记忆与泛化</a></li>
<li class="toctree-l2"><a class="reference internal" href="../2.feature_crossing/index.html">3.2. 特征交叉</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../2.feature_crossing/1.second_order.html">3.2.1. 二阶特征交叉</a></li>
<li class="toctree-l3"><a class="reference internal" href="../2.feature_crossing/2.higher_order.html">3.2.2. 高阶特征交叉</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../3.sequence.html">3.3. 序列建模</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="index.html">3.4. 多目标建模</a><ul class="current">
<li class="toctree-l3 current"><a class="current reference internal" href="#">3.4.1. 基础结构演进</a></li>
<li class="toctree-l3"><a class="reference internal" href="2.dependency_modeling.html">3.4.2. 任务依赖建模</a></li>
<li class="toctree-l3"><a class="reference internal" href="3.multi_loss_optim.html">3.4.3. 多目标损失融合</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../5.multi_scenario/index.html">3.5. 多场景建模</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../5.multi_scenario/1.multi_tower.html">3.5.1. 多塔结构</a></li>
<li class="toctree-l3"><a class="reference internal" href="../5.multi_scenario/2.dynamic_weight.html">3.5.2. 动态权重建模</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_3_rerank/index.html">4. 重排模型</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_3_rerank/1.greedy.html">4.1. 基于贪心的重排</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_3_rerank/2.personalized.html">4.2. 基于个性化的重排</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_3_rerank/3.summary.html">4.3. 本章小结</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_4_trends/index.html">5. 难点及热点研究</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_4_trends/1.debias.html">5.1. 模型去偏</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_4_trends/2.cold_start.html">5.2. 冷启动问题</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_4_trends/3.generative.html">5.3. 生成式推荐</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_4_trends/4.summary.html">5.4. 本章小结</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_5_projects/index.html">6. 项目实践</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/1.understanding.html">6.1. 赛题理解</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/2.baseline.html">6.2. Baseline</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/3.analysis.html">6.3. 数据分析</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/4.recall.html">6.4. 多路召回</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/5.feature_engineering.html">6.5. 特征工程</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_5_projects/6.ranking.html">6.6. 排序模型</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_appendix/index.html">7. Appendix</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../chapter_appendix/word2vec.html">7.1. Word2vec</a></li>
</ul>
</li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../chapter_references/references.html">参考文献</a></li>
</ul>

            </nav>
        
        </div>
    
</header>

    <div class="document">
        <div class="page-content" role="main">
        
  <section id="multi-objective-arch">
<span id="id1"></span><h1><span class="section-number">3.4.1. </span>基础结构演进<a class="headerlink" href="#multi-objective-arch" title="Permalink to this heading">¶</a></h1>
<section id="shared-bottom">
<h2><span class="section-number">3.4.1.1. </span>Shared-Bottom<a class="headerlink" href="#shared-bottom" title="Permalink to this heading">¶</a></h2>
<p>Shared-Bottom <span id="id2">(<a class="reference internal" href="../../chapter_references/references.html#id87" title="Caruana, R. (1997). Multitask learning. Machine learning, 28(1), 41–75.">Caruana, 1997</a>)</span>
模型作为多目标建模的奠基性架构，采用“共享地基+独立塔楼”的设计范式。其核心结构包含两个关键组件：</p>
<ul class="simple">
<li><p>共享底层（Shared
Bottom）：所有任务共用同一组特征转换层，负责学习跨任务的通用特征表示；</p></li>
<li><p>任务特定塔（Task-Specific
Towers）：每个任务拥有独立的顶层网络，基于共享表示学习任务特定决策边界。</p></li>
</ul>
<figure class="align-default" id="id3">
<span id="shared-bottom-model-structure"></span><a class="reference internal image-reference" href="../../_images/shared-bottom.png"><img alt="../../_images/shared-bottom.png" src="../../_images/shared-bottom.png" style="width: 500px;" /></a>
<figcaption>
<p><span class="caption-number">图3.4.1 </span><span class="caption-text">Shared-Bottom</span><a class="headerlink" href="#id3" title="Permalink to this image">¶</a></p>
</figcaption>
</figure>
<p>这种架构的数学表达可描述为：</p>
<div class="math notranslate nohighlight" id="equation-chapter-2-ranking-4-multi-objective-1-arch-0">
<span class="eqno">(3.4.1)<a class="headerlink" href="#equation-chapter-2-ranking-4-multi-objective-1-arch-0" title="Permalink to this equation">¶</a></span>\[\hat{y}_t = f_t(W_t \cdot g(W_s \mathbf{x}))\]</div>
<p>其中 <span class="math notranslate nohighlight">\(\mathbf{W}_s\)</span> 为共享层参数，<span class="math notranslate nohighlight">\(g(\cdot)\)</span>
为共享特征提取函数，<span class="math notranslate nohighlight">\(f_t(\cdot)\)</span> 为任务 <span class="math notranslate nohighlight">\(t\)</span>
的预测函数。其设计哲学建立在任务同质性假设上：不同任务共享相同的底层特征空间，仅需在顶层进行任务适配。</p>
<p>Shared-Bottom
模型在效率与泛化之间实现了良好的平衡，其核心优势主要体现在以下几点。</p>
<ol class="arabic simple">
<li><p>在参数效率方面，共享层占据了模型大部分的参数量这显著降低了模型的总参数量。</p></li>
<li><p>共享层具有正则化效应，它如同一个天然的正则化器，通过强制任务共用特征表示，有效防止了单个任务出现过拟合的情况。</p></li>
<li><p>在知识迁移方面，当任务之间存在潜在的相关性时，例如视频的点击率与完播率，共享层能够学习到通用的模式，从而提升小样本任务的泛化能力。</p></li>
</ol>
<p>然而，Shared-Bottom
模型也存在一个致命缺陷，即负迁移现象。当任务之间存在本质上的冲突时，该模型的硬共享机制会引发负迁移问题。从机制本质上来看，共享层的梯度更新方向是由所有任务共同决定的，一旦任务目标之间出现冲突，参数优化就会陷入方向性的矛盾。</p>
<p>在一些典型场景中，例如电商平台同时优化“点击率”与“客单价”时，低价商品可能会推动点击率的提升，但同时却抑制了客单价的增长；又如内容平台在平衡“内容消费深度”与“广告曝光量”时，深度阅读行为往往与广告点击行为呈负相关。</p>
<p>从数学角度来解释，假设任务<span class="math notranslate nohighlight">\(i\)</span>与任务<span class="math notranslate nohighlight">\(j\)</span>的损失梯度分别为<span class="math notranslate nohighlight">\(\nabla L_i\)</span>与<span class="math notranslate nohighlight">\(\nabla L_j\)</span>，当<span class="math notranslate nohighlight">\(\nabla L_i \cdot \nabla L_j\)</span>
&lt;
0时，共享层参数更新就会产生内在的冲突。这种冲突使得模型在处理矛盾任务时呈现出“零和博弈”的特性，即提升某一目标的性能往往需要以牺牲另一目标为代价，我们一般也称这类问题为跷跷板问题。</p>
<p><strong>Shared-Bottom代码实践</strong></p>
<p>shared-bottom模型构建代码如下，先组装输入到shared-bottom网络中的特征dnn_inputs,
经过一个shared-bottom
DNN网络，遍历创建各个任务独立的DNN塔，最后输出多个塔的预估值用于计算Loss。</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">build_shared_bottom_model</span><span class="p">(</span>
        <span class="n">feature_columns</span><span class="p">,</span>
        <span class="n">task_name_list</span><span class="p">,</span>
        <span class="n">share_dnn_units</span><span class="o">=</span><span class="p">[</span><span class="mi">128</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span>
        <span class="n">task_tower_dnn_units</span><span class="o">=</span><span class="p">[</span><span class="mi">128</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span>
        <span class="p">):</span>
    <span class="c1"># 输入层：将原始特征映射为 Keras 输入</span>
    <span class="n">input_layer_dict</span> <span class="o">=</span> <span class="n">build_input_layer</span><span class="p">(</span><span class="n">feature_columns</span><span class="p">)</span>

    <span class="c1"># 嵌入层：为各特征组创建嵌入表，得到组内嵌入向量</span>
    <span class="n">embedding_table_dict</span> <span class="o">=</span> <span class="n">build_group_feature_embedding_table_dict</span><span class="p">(</span>
        <span class="n">feature_columns</span><span class="p">,</span> <span class="n">input_layer_dict</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">&quot;embedding/&quot;</span>
    <span class="p">)</span>

    <span class="c1"># 合并嵌入：将多组嵌入拼接为共享 DNN 的输入</span>
    <span class="n">dnn_inputs</span> <span class="o">=</span> <span class="n">concat_group_embedding</span><span class="p">(</span><span class="n">embedding_table_dict</span><span class="p">,</span> <span class="s1">&#39;dnn&#39;</span><span class="p">)</span>

    <span class="c1"># 共享底座：所有任务共享的特征抽取网络（Shared Bottom）</span>
    <span class="n">shared_feature</span> <span class="o">=</span> <span class="n">DNNs</span><span class="p">(</span><span class="n">share_dnn_units</span><span class="p">)(</span><span class="n">dnn_inputs</span><span class="p">)</span>

    <span class="c1"># 任务塔：在共享特征上为每个任务构建独立塔并输出概率</span>
    <span class="n">task_outputs</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="k">for</span> <span class="n">task_name</span> <span class="ow">in</span> <span class="n">task_name_list</span><span class="p">:</span>
        <span class="n">task_logit</span> <span class="o">=</span> <span class="n">DNNs</span><span class="p">(</span><span class="n">task_tower_dnn_units</span> <span class="o">+</span> <span class="p">[</span><span class="mi">1</span><span class="p">])(</span><span class="n">shared_feature</span><span class="p">)</span>
        <span class="n">task_prob</span> <span class="o">=</span> <span class="n">PredictLayer</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;task_</span><span class="si">{</span><span class="n">task_name</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)(</span><span class="n">task_logit</span><span class="p">)</span>
        <span class="n">task_outputs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">task_prob</span><span class="p">)</span>

    <span class="c1"># 构建多任务模型：共享底座 + 多任务塔输出</span>
    <span class="n">model</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">Model</span><span class="p">(</span><span class="n">inputs</span><span class="o">=</span><span class="nb">list</span><span class="p">(</span><span class="n">input_layer_dict</span><span class="o">.</span><span class="n">values</span><span class="p">()),</span> <span class="n">outputs</span><span class="o">=</span><span class="n">task_outputs</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">model</span>
</pre></div>
</div>
<p><strong>训练和评估</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">funrec</span><span class="w"> </span><span class="kn">import</span> <span class="n">run_experiment</span>

<span class="n">run_experiment</span><span class="p">(</span><span class="s1">&#39;shared_bottom&#39;</span><span class="p">)</span>
</pre></div>
</div>
<div class="output highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+----------------+---------------+-----------------+-------------+-----------------+----------------+------------------+--------------+---------------------+--------------------+----------------------+</span>
<span class="o">|</span>   <span class="n">auc_is_click</span> <span class="o">|</span>   <span class="n">auc_is_like</span> <span class="o">|</span>   <span class="n">auc_long_view</span> <span class="o">|</span>   <span class="n">auc_macro</span> <span class="o">|</span>   <span class="n">gauc_is_click</span> <span class="o">|</span>   <span class="n">gauc_is_like</span> <span class="o">|</span>   <span class="n">gauc_long_view</span> <span class="o">|</span>   <span class="n">gauc_macro</span> <span class="o">|</span>   <span class="n">val_user_is_click</span> <span class="o">|</span>   <span class="n">val_user_is_like</span> <span class="o">|</span>   <span class="n">val_user_long_view</span> <span class="o">|</span>
<span class="o">+================+===============+=================+=============+=================+================+==================+==============+=====================+====================+======================+</span>
<span class="o">|</span>           <span class="mf">0.61</span> <span class="o">|</span>         <span class="mf">0.431</span> <span class="o">|</span>          <span class="mf">0.4364</span> <span class="o">|</span>      <span class="mf">0.4925</span> <span class="o">|</span>          <span class="mf">0.5727</span> <span class="o">|</span>         <span class="mf">0.4433</span> <span class="o">|</span>           <span class="mf">0.4438</span> <span class="o">|</span>       <span class="mf">0.4866</span> <span class="o">|</span>                 <span class="mi">928</span> <span class="o">|</span>                <span class="mi">530</span> <span class="o">|</span>                  <span class="mi">925</span> <span class="o">|</span>
<span class="o">+----------------+---------------+-----------------+-------------+-----------------+----------------+------------------+--------------+---------------------+--------------------+----------------------+</span>
</pre></div>
</div>
</section>
<section id="mmoe">
<h2><span class="section-number">3.4.1.2. </span>MMoE<a class="headerlink" href="#mmoe" title="Permalink to this heading">¶</a></h2>
<figure class="align-default" id="id4">
<span id="mmoe-model-structure"></span><a class="reference internal image-reference" href="../../_images/mmoe.png"><img alt="../../_images/mmoe.png" src="../../_images/mmoe.png" style="width: 500px;" /></a>
<figcaption>
<p><span class="caption-number">图3.4.2 </span><span class="caption-text">MMoE模型结构</span><a class="headerlink" href="#id4" title="Permalink to this image">¶</a></p>
</figcaption>
</figure>
<p>针对Shared-Bottom对相关性低的多个任务出现负迁移的现象，OMoE（OneGate
Mixture-of-Experts）将底层共享的一个Shared-Bottom模块拆分成了多个Expert，最终OMoE的输出为多个Expert的加权和，本质可以看成是专家网络和全局门控的双层结构。</p>
<p>虽然OMoE通过底层多专家融合的方式提升了特征表征的多样性，从最终的实验结果看，确实可以一定程度上缓解低相关性任务的负迁移问题，但没有彻底解决多任务冲突的问题。因为不同任务反向传播的梯度还是会直接影响底层专家网络的学习。</p>
<p>为了进一步缓解多任务冲突，MMoE（Multi-gate
Mixture-of-Experts）:cite:<cite>ma2018modeling</cite>
为每个任务配备专属门控网络，实现了门控从“全局共享”升级为“任务自适应”的方式。MMoE的数学表达式可以表示为：</p>
<div class="math notranslate nohighlight" id="equation-chapter-2-ranking-4-multi-objective-1-arch-1">
<span class="eqno">(3.4.2)<a class="headerlink" href="#equation-chapter-2-ranking-4-multi-objective-1-arch-1" title="Permalink to this equation">¶</a></span>\[\begin{split}\begin{aligned}
\mathbf{e}_k &amp;= f_k(\mathbf{x}) \\
g_t(\mathbf{x}) &amp;= \text{softmax}(\mathbf{W}_t \mathbf{x}) \\
\mathbf{h}_t &amp;= \sum_{k=1}^K g_{t,k} \cdot \mathbf{e}_k \\
\hat{y}_t &amp;= f_t(\mathbf{h}_t)
\end{aligned}\end{split}\]</div>
<p>其中 <span class="math notranslate nohighlight">\(\mathbf{x}\)</span>表示底层的特征输入，
<span class="math notranslate nohighlight">\(\mathbf{e}_k\)</span>表示第k个专家网络的输出，
<span class="math notranslate nohighlight">\(g_t(\mathbf{x})\)</span>表示第<span class="math notranslate nohighlight">\(t\)</span>个任务融合专家网络的门控向量，
<span class="math notranslate nohighlight">\(\mathbf{h}_t\)</span>表示第<span class="math notranslate nohighlight">\(t\)</span>个任务融合专家网络的输出，<span class="math notranslate nohighlight">\(\hat{y}_t\)</span>表示第<span class="math notranslate nohighlight">\(t\)</span>个任务的预测结果。</p>
<p>差异化特征融合门控网络<span class="math notranslate nohighlight">\(g_t\)</span>根据任务特性选择专家组合，例如在电商场景，CTR任务门控加权“即时兴趣”“价格敏感”专家，CVR任务门控：侧重“消费能力”“品牌忠诚”专家。</p>
<p>当任务<span class="math notranslate nohighlight">\(i\)</span>与<span class="math notranslate nohighlight">\(j\)</span>冲突时，MMoE的门控机制会让两个任务学习到不同专家的权重分布。例如，某个专家<span class="math notranslate nohighlight">\(e_m\)</span>可能在任务<span class="math notranslate nohighlight">\(i\)</span>的门控网络中获得很高的权重<span class="math notranslate nohighlight">\(g_{i,m}\)</span>，而在任务<span class="math notranslate nohighlight">\(j\)</span>的门控网络中获得很低的权重<span class="math notranslate nohighlight">\(g_{j,m}\)</span>。这样一来，专家<span class="math notranslate nohighlight">\(e_m\)</span>的参数更新主要由任务<span class="math notranslate nohighlight">\(i\)</span>的梯度决定，而任务<span class="math notranslate nohighlight">\(j\)</span>的梯度影响很小，从而实现了梯度隔离。不同任务通过选择不同的专家组合，可以各自学习到适合自己的特征表示，缓解任务冲突。</p>
<p><strong>MMoE代码实践</strong></p>
<p>MMoE模型构建代码如下，先组装输入到MoE网络中的特征dnn_inputs,
然后为每个任务创建一个门控网络输出最终融合专家网络的门控向量。最后为每个任务都创建一个任务塔，并且不同任务塔的输入都是对应任务的门控向量和多个专家网络融合后的向量。</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">build_mmoe_model</span><span class="p">(</span>
        <span class="n">feature_columns</span><span class="p">,</span>
        <span class="n">task_name_list</span><span class="p">,</span>
        <span class="n">expert_nums</span><span class="o">=</span><span class="mi">4</span><span class="p">,</span>
        <span class="n">expert_dnn_units</span><span class="o">=</span><span class="p">[</span><span class="mi">128</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span>
        <span class="n">gate_dnn_units</span><span class="o">=</span><span class="p">[</span><span class="mi">128</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span>
        <span class="n">task_tower_dnn_units</span><span class="o">=</span><span class="p">[</span><span class="mi">128</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span>
        <span class="p">):</span>
    <span class="c1"># 输入层：原始特征 → Keras 输入</span>
    <span class="n">input_layer_dict</span> <span class="o">=</span> <span class="n">build_input_layer</span><span class="p">(</span><span class="n">feature_columns</span><span class="p">)</span>

    <span class="c1"># 嵌入层：为各特征组创建嵌入表，得到组内嵌入向量</span>
    <span class="n">embedding_table_dict</span> <span class="o">=</span> <span class="n">build_group_feature_embedding_table_dict</span><span class="p">(</span>
        <span class="n">feature_columns</span><span class="p">,</span> <span class="n">input_layer_dict</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">&quot;embedding/&quot;</span>
    <span class="p">)</span>

    <span class="c1"># 合并嵌入：拼接为专家与门控的共同输入</span>
    <span class="n">dnn_inputs</span> <span class="o">=</span> <span class="n">concat_group_embedding</span><span class="p">(</span><span class="n">embedding_table_dict</span><span class="p">,</span> <span class="s1">&#39;dnn&#39;</span><span class="p">)</span>

    <span class="c1"># 共享专家：多个并行 DNN（专家）供所有任务共享</span>
    <span class="n">expert_outputs</span> <span class="o">=</span> <span class="p">[</span><span class="n">DNNs</span><span class="p">(</span><span class="n">expert_dnn_units</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;expert_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)(</span><span class="n">dnn_inputs</span><span class="p">)</span>
                      <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">expert_nums</span><span class="p">)]</span>
    <span class="c1"># 形状 [B, E, D]：按专家维度堆叠，便于后续加权求和</span>
    <span class="n">experts</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="k">lambda</span> <span class="n">xs</span><span class="p">:</span> <span class="n">tf</span><span class="o">.</span><span class="n">stack</span><span class="p">(</span><span class="n">xs</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">))(</span><span class="n">expert_outputs</span><span class="p">)</span>

    <span class="c1"># 任务门控：每个任务产生 softmax 权重，对专家加权融合</span>
    <span class="n">task_features</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="k">for</span> <span class="n">idx</span><span class="p">,</span> <span class="n">task_name</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">task_name_list</span><span class="p">):</span>
        <span class="n">gate_hidden</span> <span class="o">=</span> <span class="n">DNNs</span><span class="p">(</span><span class="n">gate_dnn_units</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;task_</span><span class="si">{</span><span class="n">idx</span><span class="si">}</span><span class="s2">_gate_mlp&quot;</span><span class="p">)(</span><span class="n">dnn_inputs</span><span class="p">)</span>
        <span class="n">gate_weights</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">Dense</span><span class="p">(</span><span class="n">expert_nums</span><span class="p">,</span> <span class="n">use_bias</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
                                             <span class="n">activation</span><span class="o">=</span><span class="s1">&#39;softmax&#39;</span><span class="p">,</span>
                                             <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;task_</span><span class="si">{</span><span class="n">idx</span><span class="si">}</span><span class="s2">_gate_softmax&quot;</span><span class="p">)(</span><span class="n">gate_hidden</span><span class="p">)</span>  <span class="c1"># [B, E]</span>
        <span class="c1"># 加权融合：einsum(&#39;be,bed-&gt;bd&#39;) == sum_e w_e * expert_e</span>
        <span class="n">task_mix</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">tf</span><span class="o">.</span><span class="n">einsum</span><span class="p">(</span><span class="s1">&#39;be,bed-&gt;bd&#39;</span><span class="p">,</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">]))([</span><span class="n">gate_weights</span><span class="p">,</span> <span class="n">experts</span><span class="p">])</span>
        <span class="n">task_features</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">task_mix</span><span class="p">)</span>

    <span class="c1"># 任务塔：基于融合特征为每个任务建塔并输出概率</span>
    <span class="n">task_outputs</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="k">for</span> <span class="n">task_name</span><span class="p">,</span> <span class="n">task_feat</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">task_name_list</span><span class="p">,</span> <span class="n">task_features</span><span class="p">):</span>
        <span class="n">task_logit</span> <span class="o">=</span> <span class="n">DNNs</span><span class="p">(</span><span class="n">task_tower_dnn_units</span> <span class="o">+</span> <span class="p">[</span><span class="mi">1</span><span class="p">])(</span><span class="n">task_feat</span><span class="p">)</span>
        <span class="n">task_prob</span> <span class="o">=</span> <span class="n">PredictLayer</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;task_</span><span class="si">{</span><span class="n">task_name</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)(</span><span class="n">task_logit</span><span class="p">)</span>
        <span class="n">task_outputs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">task_prob</span><span class="p">)</span>

    <span class="c1"># 构建模型：共享专家 + 任务门控 + 任务塔</span>
    <span class="n">model</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">Model</span><span class="p">(</span><span class="n">inputs</span><span class="o">=</span><span class="nb">list</span><span class="p">(</span><span class="n">input_layer_dict</span><span class="o">.</span><span class="n">values</span><span class="p">()),</span> <span class="n">outputs</span><span class="o">=</span><span class="n">task_outputs</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">model</span>
</pre></div>
</div>
<p><strong>训练和评估</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">run_experiment</span><span class="p">(</span><span class="s1">&#39;mmoe&#39;</span><span class="p">)</span>
</pre></div>
</div>
<div class="output highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+----------------+---------------+-----------------+-------------+-----------------+----------------+------------------+--------------+---------------------+--------------------+----------------------+</span>
<span class="o">|</span>   <span class="n">auc_is_click</span> <span class="o">|</span>   <span class="n">auc_is_like</span> <span class="o">|</span>   <span class="n">auc_long_view</span> <span class="o">|</span>   <span class="n">auc_macro</span> <span class="o">|</span>   <span class="n">gauc_is_click</span> <span class="o">|</span>   <span class="n">gauc_is_like</span> <span class="o">|</span>   <span class="n">gauc_long_view</span> <span class="o">|</span>   <span class="n">gauc_macro</span> <span class="o">|</span>   <span class="n">val_user_is_click</span> <span class="o">|</span>   <span class="n">val_user_is_like</span> <span class="o">|</span>   <span class="n">val_user_long_view</span> <span class="o">|</span>
<span class="o">+================+===============+=================+=============+=================+================+==================+==============+=====================+====================+======================+</span>
<span class="o">|</span>         <span class="mf">0.6036</span> <span class="o">|</span>        <span class="mf">0.4203</span> <span class="o">|</span>          <span class="mf">0.4586</span> <span class="o">|</span>      <span class="mf">0.4942</span> <span class="o">|</span>          <span class="mf">0.5728</span> <span class="o">|</span>         <span class="mf">0.4362</span> <span class="o">|</span>           <span class="mf">0.4759</span> <span class="o">|</span>        <span class="mf">0.495</span> <span class="o">|</span>                 <span class="mi">928</span> <span class="o">|</span>                <span class="mi">530</span> <span class="o">|</span>                  <span class="mi">925</span> <span class="o">|</span>
<span class="o">+----------------+---------------+-----------------+-------------+-----------------+----------------+------------------+--------------+---------------------+--------------------+----------------------+</span>
</pre></div>
</div>
</section>
<section id="ple">
<h2><span class="section-number">3.4.1.3. </span>PLE<a class="headerlink" href="#ple" title="Permalink to this heading">¶</a></h2>
<p>MMoE
通过为每个任务配备专属门控网络，在一定程度上缓解了多任务冲突问题。专属门控网络能够根据任务特性选择专家组合，从而使得不同任务可以关注不同的特征表示。但其架构仍存在一个根本性局限：所有专家对所有任务门控可见。这种“软隔离”设计在实践中仍面临两大挑战：</p>
<ol class="arabic simple">
<li><p>负迁移未根除：</p>
<ul class="simple">
<li><p>干扰路径未切断：即使某个专家（如<span class="math notranslate nohighlight">\(e_m\)</span>）被任务<span class="math notranslate nohighlight">\(i\)</span>的门控高度加权而被任务<span class="math notranslate nohighlight">\(j\)</span>的门控忽略，任务<span class="math notranslate nohighlight">\(j\)</span>的梯度在反向传播时仍会流经<span class="math notranslate nohighlight">\(e_m\)</span>（因为<span class="math notranslate nohighlight">\(e_m\)</span>是任务<span class="math notranslate nohighlight">\(j\)</span>门控的可选项）。当任务冲突强烈时，这种“潜在通路”仍可能导致共享表征被污染。</p></li>
<li><p>专家角色模糊：MMoE缺乏机制强制专家明确分工。一个专家可能同时承载共享信息和多个任务的特定信息，成为冲突的“重灾区”。尤其在任务相关性低时，这种耦合会加剧负迁移。</p></li>
</ul>
</li>
<li><p>门控决策负担重：</p>
<ul class="simple">
<li><p>每个任务的门控需要在所有<span class="math notranslate nohighlight">\(K\)</span>个专家上进行权重分配。当专家数量增加（通常需扩大<span class="math notranslate nohighlight">\(K\)</span>以提升模型能力）时，门控网络面临高维决策问题，易导致训练不稳定或陷入次优解。</p></li>
<li><p>门控需要“费力”地从包含混杂信息（共享+所有任务特定）
的专家池中筛选有用信息，增加了学习难度。</p></li>
</ul>
</li>
</ol>
<p>为解决上述问题，cite:<code class="docutils literal notranslate"><span class="pre">tang2020progressive</span></code> 提出提出了CGC（Customized
Gate
Control）结构，其核心思想是通过硬性结构约束，显式分离共享知识与任务特定知识：</p>
<figure class="align-default" id="id5">
<span id="cgc-model-structure"></span><a class="reference internal image-reference" href="../../_images/cgc.png"><img alt="../../_images/cgc.png" src="../../_images/cgc.png" style="width: 350px;" /></a>
<figcaption>
<p><span class="caption-number">图3.4.3 </span><span class="caption-text">CGC网络结构</span><a class="headerlink" href="#id5" title="Permalink to this image">¶</a></p>
</figcaption>
</figure>
<p><strong>专家职责强制分离</strong>：</p>
<ul class="simple">
<li><p>共享专家
(C-Experts)：一组专家仅负责学习所有任务的共性知识。设其数量为<span class="math notranslate nohighlight">\(M\)</span>，输出为<span class="math notranslate nohighlight">\({\mathbf{c}_1, \mathbf{c}_2, ..., \mathbf{c}_M}\)</span>。</p></li>
<li><p>任务专家
(T-Experts)：每个任务<span class="math notranslate nohighlight">\(t\)</span>拥有自己专属的专家组，仅负责学习该任务特有的知识或模式。设任务<span class="math notranslate nohighlight">\(t\)</span>的专属专家数量为<span class="math notranslate nohighlight">\(N_t\)</span>，输出为<span class="math notranslate nohighlight">\({\mathbf{t}_t^1, \mathbf{t}_t^2, ..., \mathbf{t}_t^{N_t}}\)</span>。</p></li>
</ul>
<p><strong>任务专属门控的输入限制</strong>：</p>
<ul class="simple">
<li><p>任务<span class="math notranslate nohighlight">\(t\)</span>的门控<span class="math notranslate nohighlight">\(g_t\)</span>的输入被严格限制为：共享专家输出
(<span class="math notranslate nohighlight">\({\mathbf{c}k}{k=1}^M\)</span>) + 本任务专属专家输出
(<span class="math notranslate nohighlight">\({\mathbf{t}t^j}{j=1}^{N_t}\)</span>)。</p></li>
<li><p>物理切断干扰路径：任务<span class="math notranslate nohighlight">\(t\)</span>的门控完全无法访问其他任务<span class="math notranslate nohighlight">\(s (s \neq t)\)</span>的专属专家<span class="math notranslate nohighlight">\({\mathbf{t}_s^j}\)</span>。同样，任务<span class="math notranslate nohighlight">\(s\)</span>的梯度绝不会更新任务<span class="math notranslate nohighlight">\(t\)</span>的专属专家参数。</p></li>
</ul>
<p>CGC门控的计算如下：</p>
<div class="math notranslate nohighlight" id="equation-chapter-2-ranking-4-multi-objective-1-arch-2">
<span class="eqno">(3.4.3)<a class="headerlink" href="#equation-chapter-2-ranking-4-multi-objective-1-arch-2" title="Permalink to this equation">¶</a></span>\[g_t(\mathbf{x}) = \text{softmax}\Big(\mathbf{W}_t \cdot \mathbf{x} + \mathbf{b}_t\Big) \quad\]</div>
<div class="math notranslate nohighlight" id="equation-chapter-2-ranking-4-multi-objective-1-arch-3">
<span class="eqno">(3.4.4)<a class="headerlink" href="#equation-chapter-2-ranking-4-multi-objective-1-arch-3" title="Permalink to this equation">¶</a></span>\[\mathbf{h}_t = \sum_{k=1}^{M} g_{t,k} \cdot \mathbf{c}_k + \sum_{j=1}^{N_t} g_{t, M+j} \cdot \mathbf{t}_t^j\]</div>
<div class="math notranslate nohighlight" id="equation-chapter-2-ranking-4-multi-objective-1-arch-4">
<span class="eqno">(3.4.5)<a class="headerlink" href="#equation-chapter-2-ranking-4-multi-objective-1-arch-4" title="Permalink to this equation">¶</a></span>\[\hat{y}_t = f_t(\mathbf{h}_t)\]</div>
<p>其中：</p>
<ul class="simple">
<li><p><span class="math notranslate nohighlight">\(\mathbf{W}_t, \mathbf{b}_t\)</span>：任务<span class="math notranslate nohighlight">\(t\)</span>门控的参数。</p></li>
<li><p><span class="math notranslate nohighlight">\(g_{t,k}\)</span>：分配给第<span class="math notranslate nohighlight">\(k\)</span>个共享专家的权重。</p></li>
<li><p><span class="math notranslate nohighlight">\(g_{t, M+j}\)</span>：分配给任务<span class="math notranslate nohighlight">\(t\)</span>第<span class="math notranslate nohighlight">\(j\)</span>个专属专家的权重。</p></li>
</ul>
<p><strong>CGC核心代码为：</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">cgc_net</span><span class="p">(</span>
        <span class="n">input_list</span><span class="p">,</span>
        <span class="n">task_num</span><span class="p">,</span>
        <span class="n">task_expert_num</span><span class="p">,</span>
        <span class="n">shared_expert_num</span><span class="p">,</span>
        <span class="n">task_expert_dnn_units</span><span class="p">,</span>
        <span class="n">shared_expert_dnn_units</span><span class="p">,</span>
        <span class="n">task_gate_dnn_units</span><span class="p">,</span>
        <span class="n">shared_gate_dnn_units</span><span class="p">,</span>
        <span class="n">leval_name</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
        <span class="n">is_last</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
<span class="w">    </span><span class="sd">&quot;&quot;&quot;CGC（共享专家 + 任务门控）核心结构（简化版）</span>
<span class="sd">    - 每个任务：拥有若干 Task-Experts；</span>
<span class="sd">    - 全局：拥有若干 Shared-Experts；</span>
<span class="sd">    - 每个任务 Gate 产生 softmax 权重，对其 Task-Experts 与 Shared-Experts 加权融合；</span>
<span class="sd">    - 若非最后一层：再用 Shared-Gate 融合所有任务的 Task-Experts 与 Shared-Experts，供下一层共享使用。</span>
<span class="sd">    input_list：为方便处理，给每个任务复制一份相同输入，最后一个为共享输入。</span>
<span class="sd">    &quot;&quot;&quot;</span>

    <span class="c1"># 任务专家：每个任务创建 task_expert_num 个专家</span>
    <span class="n">task_expert_list</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">task_num</span><span class="p">):</span>
        <span class="n">task_expert_list</span><span class="o">.</span><span class="n">append</span><span class="p">([</span>
            <span class="n">DNNs</span><span class="p">(</span><span class="n">task_expert_dnn_units</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="n">leval_name</span><span class="si">}</span><span class="s2">_task_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">_expert_</span><span class="si">{</span><span class="n">j</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)(</span><span class="n">input_list</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
            <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">task_expert_num</span><span class="p">)</span>
        <span class="p">])</span>

    <span class="c1"># 共享专家：创建 shared_expert_num 个专家（共享输入使用 input_list[-1]）</span>
    <span class="n">shared_expert_list</span> <span class="o">=</span> <span class="p">[</span>
        <span class="n">DNNs</span><span class="p">(</span><span class="n">shared_expert_dnn_units</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="n">leval_name</span><span class="si">}</span><span class="s2">_shared_expert_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)(</span><span class="n">input_list</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
        <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">shared_expert_num</span><span class="p">)</span>
    <span class="p">]</span>

    <span class="c1"># 任务门控与融合：对当前任务的（Task + Shared）专家集合进行 softmax 加权求和</span>
    <span class="n">cgc_outputs</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">fusion_expert_num</span> <span class="o">=</span> <span class="n">task_expert_num</span> <span class="o">+</span> <span class="n">shared_expert_num</span>
    <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">task_num</span><span class="p">):</span>
        <span class="n">cur_experts</span> <span class="o">=</span> <span class="n">task_expert_list</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">+</span> <span class="n">shared_expert_list</span>
        <span class="n">experts</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="k">lambda</span> <span class="n">xs</span><span class="p">:</span> <span class="n">tf</span><span class="o">.</span><span class="n">stack</span><span class="p">(</span><span class="n">xs</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">))(</span><span class="n">cur_experts</span><span class="p">)</span>  <span class="c1"># [B, E, D]</span>

        <span class="n">gate_hidden</span> <span class="o">=</span> <span class="n">DNNs</span><span class="p">(</span><span class="n">task_gate_dnn_units</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="n">leval_name</span><span class="si">}</span><span class="s2">_task_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">_gate&quot;</span><span class="p">)(</span><span class="n">input_list</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
        <span class="n">gate_weights</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">Dense</span><span class="p">(</span><span class="n">fusion_expert_num</span><span class="p">,</span> <span class="n">use_bias</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">&#39;softmax&#39;</span><span class="p">)(</span><span class="n">gate_hidden</span><span class="p">)</span>  <span class="c1"># [B, E]</span>

        <span class="c1"># 加权融合：einsum(&#39;be,bed-&gt;bd&#39;) == sum_e w_e * expert_e</span>
        <span class="n">fused</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">tf</span><span class="o">.</span><span class="n">einsum</span><span class="p">(</span><span class="s1">&#39;be,bed-&gt;bd&#39;</span><span class="p">,</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">]))([</span><span class="n">gate_weights</span><span class="p">,</span> <span class="n">experts</span><span class="p">])</span>
        <span class="n">cgc_outputs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">fused</span><span class="p">)</span>

    <span class="c1"># 若非最后一层：共享门控融合所有任务专家与共享专家，作为下一层共享输入</span>
    <span class="k">if</span> <span class="ow">not</span> <span class="n">is_last</span><span class="p">:</span>
        <span class="c1"># 展平所有任务的专家 + 共享专家</span>
        <span class="n">all_task_experts</span> <span class="o">=</span> <span class="p">[</span><span class="n">e</span> <span class="k">for</span> <span class="n">task</span> <span class="ow">in</span> <span class="n">task_expert_list</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">task</span><span class="p">]</span>
        <span class="n">cur_experts</span> <span class="o">=</span> <span class="n">all_task_experts</span> <span class="o">+</span> <span class="n">shared_expert_list</span>
        <span class="n">experts_all</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="k">lambda</span> <span class="n">xs</span><span class="p">:</span> <span class="n">tf</span><span class="o">.</span><span class="n">stack</span><span class="p">(</span><span class="n">xs</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">))(</span><span class="n">cur_experts</span><span class="p">)</span>  <span class="c1"># [B, E_all, D]</span>
        <span class="n">cur_expert_num</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">cur_experts</span><span class="p">)</span>

        <span class="n">shared_gate_hidden</span> <span class="o">=</span> <span class="n">DNNs</span><span class="p">(</span><span class="n">shared_gate_dnn_units</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="n">leval_name</span><span class="si">}</span><span class="s2">_shared_gate&quot;</span><span class="p">)(</span><span class="n">input_list</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
        <span class="n">shared_gate_weights</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">Dense</span><span class="p">(</span><span class="n">cur_expert_num</span><span class="p">,</span> <span class="n">use_bias</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">&#39;softmax&#39;</span><span class="p">)(</span><span class="n">shared_gate_hidden</span><span class="p">)</span>  <span class="c1"># [B, E_all]</span>
        <span class="n">shared_fused</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">tf</span><span class="o">.</span><span class="n">einsum</span><span class="p">(</span><span class="s1">&#39;be,bed-&gt;bd&#39;</span><span class="p">,</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">]))([</span><span class="n">shared_gate_weights</span><span class="p">,</span> <span class="n">experts_all</span><span class="p">])</span>
        <span class="n">cgc_outputs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">shared_fused</span><span class="p">)</span>

    <span class="k">return</span> <span class="n">cgc_outputs</span>
</pre></div>
</div>
<p>CGC解决了知识分离的核心问题，但其本质是单层结构，表征学习深度有限。受深度神经网络逐层抽象特征的启发，PLE
(Progressive Layered Extraction)
将多个CGC单元纵向堆叠，形成深层架构，实现渐进式知识提取与融合。</p>
<figure class="align-default" id="id6">
<span id="ple-model-structure"></span><a class="reference internal image-reference" href="../../_images/ple.png"><img alt="../../_images/ple.png" src="../../_images/ple.png" style="width: 400px;" /></a>
<figcaption>
<p><span class="caption-number">图3.4.4 </span><span class="caption-text">PLE网络结构</span><a class="headerlink" href="#id6" title="Permalink to this image">¶</a></p>
</figcaption>
</figure>
<ul class="simple">
<li><p>第1层 (输入层 CGC)：</p>
<ul>
<li><p>输入：原始特征 <span class="math notranslate nohighlight">\(\mathbf{x}\)</span>。</p></li>
<li><p>结构：一个标准的CGC模块（包含 <span class="math notranslate nohighlight">\(M^{(1)}\)</span>个C-Experts,
每个任务<span class="math notranslate nohighlight">\(t\)</span>有 <span class="math notranslate nohighlight">\(N_t^{(1)}\)</span>个T-Experts,
以及对应的门控 <span class="math notranslate nohighlight">\(g_t^{(1)}\)</span>）。</p></li>
<li><p>输出：每个任务获得初步融合表示
<span class="math notranslate nohighlight">\(\mathbf{h}_t^{(1)}\)</span>（或更常见的是，该层所有专家（C+T）的输出被拼接/收集起来作为下一层输入）。</p></li>
</ul>
</li>
<li><p>第<span class="math notranslate nohighlight">\(l\)</span>层 (<span class="math notranslate nohighlight">\(l \geq 2\)</span>) CGC：</p>
<ul>
<li><p>输入关键点：第<span class="math notranslate nohighlight">\(l\)</span>层的输入是第<span class="math notranslate nohighlight">\(l-1\)</span>层所有专家（包括所有C-Experts和所有任务的T-Experts）的输出。设第<span class="math notranslate nohighlight">\(l-1\)</span>层总专家数为
<span class="math notranslate nohighlight">\(E^{(l-1)}\)</span>，则输入向量维度相应增加。</p></li>
<li><p>结构：一个新的CGC模块（包含 <span class="math notranslate nohighlight">\(M^{(l)}\)</span>个C-Experts,
每个任务<span class="math notranslate nohighlight">\(t\)</span>有 <span class="math notranslate nohighlight">\(N_t^{(l)}\)</span>个T-Experts, 以及门控
<span class="math notranslate nohighlight">\(g_t^{(l)}\)</span>）。</p></li>
<li><p>处理：在本层输入特征（即上一层更丰富的表征）上，再次进行显式的知识分离（新的C-Experts学习更深层共享模式，新的T-Experts学习更深层任务特定模式）和融合（通过新的门控）。</p></li>
<li><p>输出：任务<span class="math notranslate nohighlight">\(t\)</span>的当前层表示 <span class="math notranslate nohighlight">\(\mathbf{h}_t^{(l)}\)</span>
或收集所有专家输出。</p></li>
</ul>
</li>
<li><p>输出层 (第<span class="math notranslate nohighlight">\(L\)</span>层)：</p>
<ul>
<li><p>最后一层(<span class="math notranslate nohighlight">\(L\)</span>)各任务的门控输出 <span class="math notranslate nohighlight">\(\mathbf{h}_t^{(L)}\)</span>
送入各自的任务专属塔网络（Tower）<span class="math notranslate nohighlight">\(f_t\)</span>，得到最终预测
<span class="math notranslate nohighlight">\(\hat{y}_t = f_t(\mathbf{h}_t^{(L)})\)</span>。</p></li>
</ul>
</li>
</ul>
<p><strong>PLE模型的核心代码：</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">build_ple_model</span><span class="p">(</span>
        <span class="n">feature_columns</span><span class="p">,</span>
        <span class="n">task_name_list</span><span class="p">,</span>
        <span class="n">ple_level_nums</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span>
        <span class="n">task_expert_num</span><span class="o">=</span><span class="mi">4</span><span class="p">,</span>
        <span class="n">shared_expert_num</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span>
        <span class="n">task_expert_dnn_units</span><span class="o">=</span><span class="p">[</span><span class="mi">128</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span>
        <span class="n">shared_expert_dnn_units</span><span class="o">=</span><span class="p">[</span><span class="mi">128</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span>
        <span class="n">task_gate_dnn_units</span><span class="o">=</span><span class="p">[</span><span class="mi">128</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span>
        <span class="n">shared_gate_dnn_units</span><span class="o">=</span><span class="p">[</span><span class="mi">128</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span>
        <span class="n">task_tower_dnn_units</span><span class="o">=</span><span class="p">[</span><span class="mi">128</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span>
        <span class="p">):</span>
    <span class="c1"># 1) 输入与嵌入：构建输入层/分组嵌入，拼接为 PLE 的共享输入</span>
    <span class="n">input_layer_dict</span> <span class="o">=</span> <span class="n">build_input_layer</span><span class="p">(</span><span class="n">feature_columns</span><span class="p">)</span>
    <span class="n">group_embedding_feature_dict</span> <span class="o">=</span> <span class="n">build_group_feature_embedding_table_dict</span><span class="p">(</span>
        <span class="n">feature_columns</span><span class="p">,</span> <span class="n">input_layer_dict</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">&quot;embedding/&quot;</span>
    <span class="p">)</span>
    <span class="n">dnn_inputs</span> <span class="o">=</span> <span class="n">concat_group_embedding</span><span class="p">(</span><span class="n">group_embedding_feature_dict</span><span class="p">,</span> <span class="s1">&#39;dnn&#39;</span><span class="p">)</span>

    <span class="c1"># 2) 级联 PLE（CGC）层：每层包含“任务专家 + 共享专家 + 门控”，最后一层仅输出任务特征</span>
    <span class="n">task_num</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">task_name_list</span><span class="p">)</span>
    <span class="n">ple_input_list</span> <span class="o">=</span> <span class="p">[</span><span class="n">dnn_inputs</span><span class="p">]</span> <span class="o">*</span> <span class="p">(</span><span class="n">task_num</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span>  <span class="c1"># 前 task_num 为各任务输入，末尾为共享输入</span>
    <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">ple_level_nums</span><span class="p">):</span>
        <span class="n">is_last</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span> <span class="o">==</span> <span class="n">ple_level_nums</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
        <span class="n">ple_input_list</span> <span class="o">=</span> <span class="n">cgc_net</span><span class="p">(</span>
            <span class="n">ple_input_list</span><span class="p">,</span>
            <span class="n">task_num</span><span class="p">,</span>
            <span class="n">task_expert_num</span><span class="p">,</span>
            <span class="n">shared_expert_num</span><span class="p">,</span>
            <span class="n">task_expert_dnn_units</span><span class="p">,</span>
            <span class="n">shared_expert_dnn_units</span><span class="p">,</span>
            <span class="n">task_gate_dnn_units</span><span class="p">,</span>
            <span class="n">shared_gate_dnn_units</span><span class="p">,</span>
            <span class="n">leval_name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;cgc_level_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">,</span>
            <span class="n">is_last</span><span class="o">=</span><span class="n">is_last</span>
        <span class="p">)</span>

    <span class="c1"># 3) 任务塔与输出：将各任务特征送入塔 DNN，得到每个任务的概率输出</span>
    <span class="n">task_output_list</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">task_num</span><span class="p">):</span>
        <span class="n">task_logit</span> <span class="o">=</span> <span class="n">DNNs</span><span class="p">(</span><span class="n">task_tower_dnn_units</span> <span class="o">+</span> <span class="p">[</span><span class="mi">1</span><span class="p">])(</span><span class="n">ple_input_list</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
        <span class="n">task_prob</span> <span class="o">=</span> <span class="n">PredictLayer</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;task_&quot;</span> <span class="o">+</span> <span class="n">task_name_list</span><span class="p">[</span><span class="n">i</span><span class="p">])(</span><span class="n">task_logit</span><span class="p">)</span>
        <span class="n">task_output_list</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">task_prob</span><span class="p">)</span>

    <span class="c1"># 4) 构建模型：输入为所有原始输入层，输出为各任务概率</span>
    <span class="n">model</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">keras</span><span class="o">.</span><span class="n">Model</span><span class="p">(</span><span class="n">inputs</span><span class="o">=</span><span class="nb">list</span><span class="p">(</span><span class="n">input_layer_dict</span><span class="o">.</span><span class="n">values</span><span class="p">()),</span> <span class="n">outputs</span><span class="o">=</span><span class="n">task_output_list</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">model</span>
</pre></div>
</div>
<p><strong>训练和评估</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">run_experiment</span><span class="p">(</span><span class="s1">&#39;ple&#39;</span><span class="p">)</span>
</pre></div>
</div>
<div class="output highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+----------------+---------------+-----------------+-------------+-----------------+----------------+------------------+--------------+---------------------+--------------------+----------------------+</span>
<span class="o">|</span>   <span class="n">auc_is_click</span> <span class="o">|</span>   <span class="n">auc_is_like</span> <span class="o">|</span>   <span class="n">auc_long_view</span> <span class="o">|</span>   <span class="n">auc_macro</span> <span class="o">|</span>   <span class="n">gauc_is_click</span> <span class="o">|</span>   <span class="n">gauc_is_like</span> <span class="o">|</span>   <span class="n">gauc_long_view</span> <span class="o">|</span>   <span class="n">gauc_macro</span> <span class="o">|</span>   <span class="n">val_user_is_click</span> <span class="o">|</span>   <span class="n">val_user_is_like</span> <span class="o">|</span>   <span class="n">val_user_long_view</span> <span class="o">|</span>
<span class="o">+================+===============+=================+=============+=================+================+==================+==============+=====================+====================+======================+</span>
<span class="o">|</span>         <span class="mf">0.5975</span> <span class="o">|</span>        <span class="mf">0.4191</span> <span class="o">|</span>          <span class="mf">0.4474</span> <span class="o">|</span>       <span class="mf">0.488</span> <span class="o">|</span>           <span class="mf">0.574</span> <span class="o">|</span>         <span class="mf">0.4434</span> <span class="o">|</span>           <span class="mf">0.4561</span> <span class="o">|</span>       <span class="mf">0.4912</span> <span class="o">|</span>                 <span class="mi">928</span> <span class="o">|</span>                <span class="mi">530</span> <span class="o">|</span>                  <span class="mi">925</span> <span class="o">|</span>
<span class="o">+----------------+---------------+-----------------+-------------+-----------------+----------------+------------------+--------------+---------------------+--------------------+----------------------+</span>
</pre></div>
</div>
</section>
</section>


        </div>
        <div class="side-doc-outline">
            <div class="side-doc-outline--content"> 
<div class="localtoc">
    <p class="caption">
      <span class="caption-text">Table Of Contents</span>
    </p>
    <ul>
<li><a class="reference internal" href="#">3.4.1. 基础结构演进</a><ul>
<li><a class="reference internal" href="#shared-bottom">3.4.1.1. Shared-Bottom</a></li>
<li><a class="reference internal" href="#mmoe">3.4.1.2. MMoE</a></li>
<li><a class="reference internal" href="#ple">3.4.1.3. PLE</a></li>
</ul>
</li>
</ul>

</div>
            </div>
        </div>

      <div class="clearer"></div>
    </div><div class="pagenation">
     <a id="button-prev" href="index.html" class="mdl-button mdl-js-button mdl-js-ripple-effect mdl-button--colored" role="botton" accesskey="P">
         <i class="pagenation-arrow-L fas fa-arrow-left fa-lg"></i>
         <div class="pagenation-text">
            <span class="pagenation-direction">Previous</span>
            <div>3.4. 多目标建模</div>
         </div>
     </a>
     <a id="button-next" href="2.dependency_modeling.html" class="mdl-button mdl-js-button mdl-js-ripple-effect mdl-button--colored" role="botton" accesskey="N">
         <i class="pagenation-arrow-R fas fa-arrow-right fa-lg"></i>
        <div class="pagenation-text">
            <span class="pagenation-direction">Next</span>
            <div>3.4.2. 任务依赖建模</div>
        </div>
     </a>
  </div>
        
        </main>
    </div>
  </body>
</html>